A Boundedness Theoretical Analysis for GrADPDesign : A Case Study on Maze Navigation Report

A Boundedness Theoretical Analysis for GrADPDesign: A Case Study on Maze Navigation Report Title A new theoretical analysis towards the goal representation adaptive dynamic programming (GrADP) design proposed in [1], [2] is investigated in this paper. Unlike the proofs of convergence for adaptive dynamic programming (ADP) in literature, here we provide a new insight for the error bound between the estimated value function and the expected value function. Then we employ the critic network in GrADP approach to approximate the Q value function, and use the action network to provide the control policy. The goal network is adopted to provide the internal reinforcement signal for the critic network over time. Finally, we illustrate that the estimated Q value function is close to the expected value function in an arbitrary small bound on the maze navigation example. Conference Name: Proc. Int. Joint Conf. Neural Networks (IJCNN'15) Conference Date: July 13, 2015 A Boundedness Theoretical Analysis for GrADP Design: A Case Study on Maze Navigation Zhen Ni, Xiangnan Zhong, and Haibo He Department of Electrical, Computer, and Biomedical Engineering University of Rhode Island Kingston, RI, USA 02881 Email: {ni,xzhong,he}@ele.uri.edu Abstract—A new theoretical analysis towards the goal representation adaptive dynamic programming (GrADP) design proposed in [1], [2] is investigated in this paper. Unlike the proofs of convergence for adaptive dynamic programming (ADP) in literature, here we provide a new insight for the error bound between the estimated value function and the expected value function. Then we employ the critic network in GrADP approach to approximate the Q value function, and use the action network to provide the control policy. The goal network is adopted to provide the internal reinforcement signal for the critic network over time. Finally, we illustrate that the estimated Q value function is close to the expected value function in an arbitrary small bound on the maze navigation example.